
 <!DOCTYPE HTML>
<html>
<head><meta name="generator" content="Hexo 3.9.0">
  <meta charset="UTF-8">
  
    <title>kubelet config和认证 | Zong&#39;s blog</title>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=3, minimum-scale=1">
    
    <meta name="author" content="Zong">
    
    <meta name="description" content="Kubelet Configurationkubelet是安装在k8s节点上负责启动销毁容器的重要组件，其启动参数我一直是通过systemctl使用参数形式传入的，但是根据文档和其自身help的说明来看，很多参数标注了DEPRECATED。

例如：      –fail-swap-onMakes ">
    
    
    
    
    
    <link rel="icon" href="/img/favicon.ico">
    
    
    <link rel="apple-touch-icon" href="/img/pacman.jpg">
    <link rel="apple-touch-icon-precomposed" href="/img/pacman.jpg">
    
    <link rel="stylesheet" href="/css/style.css">
</head>
</html>
  <body>
    <header>
      <div>
		
			<div id="imglogo">
				<a href="/"><img src="/img/logo.svg" alt="Zong&#39;s blog" title="Zong&#39;s blog"/></a>
			</div>
			
			<div id="textlogo">
				<h1 class="site-name"><a href="/" title="Zong&#39;s blog">Zong&#39;s blog</a></h1>
				<h2 class="blog-motto">日常积累，技术分享</h2>
			</div>
			<div class="navbar"><a class="navbutton navmobile" href="#" title="Menu">
			</a></div>
			<nav class="animated">
				<ul>
					<ul>
					 
						<li><a href="/">Home</a></li>
					
						<li><a href="/archives">Archives</a></li>
					
						<li><a href="/categories/运维">运维</a></li>
					
						<li><a href="/categories/容器架构">容器架构</a></li>
					
					<li>
					
					<form class="search" action="//baidu.com/s" method="get" accept-charset="utf-8">
						<label>Search</label>
						<input type="text" id="search" name="wd" autocomplete="off" maxlength="20" placeholder="Search" />
                        <input name=tn type=hidden value="bds">
                        <input name=cl type=hidden value="3">
                        <input name=ct type=hidden value="2097152">
						<input type="hidden" name="si" value="www.lstop.pub">
					</form>
					
					</li>
				</ul>
			</nav>			
</div>

    </header>
    <div id="container">
      <div id="main" class="post" itemscope itemprop="blogPost">
	<article itemprop="articleBody"> 
		<header class="article-info clearfix">
  <h1 itemprop="name">
    
      <a href="/2020/05/19/kubelet-config和认证/" title="kubelet config和认证" itemprop="url">kubelet config和认证</a>
  </h1>
  <p class="article-author">By
    
      <a href="http://www.lstop.pub" title="Zong">Zong</a>
    </p>
  <p class="article-time">
    <time datetime="2020-05-19T06:04:30.000Z" itemprop="datePublished">2020-05-19</time>
    
  </p>
</header>

	<div class="article-content">
		
		
		<div id="toc" class="toc-article">
			<strong class="toc-title">Contents</strong>
		<ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#Kubelet-Configuration"><span class="toc-number">1.</span> <span class="toc-text">Kubelet Configuration</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#kubelet-认证"><span class="toc-number">2.</span> <span class="toc-text">kubelet 认证</span></a></li></ol>
		</div>
		
		<h1 id="Kubelet-Configuration"><a href="#Kubelet-Configuration" class="headerlink" title="Kubelet Configuration"></a>Kubelet Configuration</h1><p>kubelet是安装在k8s节点上负责启动销毁容器的重要组件，其启动参数我一直是通过systemctl使用参数形式传入的，但是根据文档和其自身help的说明来看，很多参数标注了DEPRECATED。</p>
<blockquote>
<p>例如：      –fail-swap-on<br>Makes the Kubelet fail to start if swap is enabled on the node.  (default true) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet’s –config flag. See <a href="https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/" target="_blank" rel="noopener">https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/</a> for more information.)</p>
</blockquote>
<p>这些参数已经不推荐直接用命令传参的方式使用了，而是引入了Kubelet Configuration。<a href="https://kubernetes.io/zh/docs/tasks/administer-cluster/kubelet-config-file/" target="_blank" rel="noopener">官方说明</a><br>以下是一个例子</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">kind: KubeletConfiguration</span><br><span class="line">apiVersion: kubelet.config.k8s.io/v1beta1</span><br><span class="line">port: 10250</span><br><span class="line">readOnlyPort: 10255</span><br><span class="line">cgroupDriver: cgroupfs</span><br><span class="line">clusterDNS: </span><br><span class="line">- 192.168.0.2</span><br><span class="line">clusterDomain: k8s.local</span><br><span class="line">failSwapOn: true</span><br><span class="line">authentication:</span><br><span class="line">  anonymous:</span><br><span class="line">    enabled: false</span><br><span class="line">  x509:</span><br><span class="line">    clientCAFile: /etc/kubernetes/pki/kubelet-ca.pem</span><br><span class="line">staticPodPath: /etc/kubernetes/staticPods</span><br><span class="line">imageGcHighThreshold: 70</span><br><span class="line">imageGcLowThreshold: 50</span><br><span class="line">featureGates: </span><br><span class="line">  RotateKubeletClientCertificate: true</span><br><span class="line">  RotateKubeletServerCertificate: true</span><br><span class="line">rotateCertificates: true</span><br></pre></td></tr></table></figure>

<p>使用–config参数指定这个kubelet config文件，原本的命令参数变成：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">KUBELET_ARGS=&quot;--bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig \</span><br><span class="line">--kubeconfig=/etc/kubernetes/kubelet.kubeconfig \</span><br><span class="line">--cert-dir=/etc/kubernetes/pki \</span><br><span class="line">--network-plugin=cni \</span><br><span class="line">--node-labels=node.kubernetes.io/role=k8s-node \</span><br><span class="line">--pod-infra-container-image=ccr.ccs.tencentyun.com/google_container/pause-amd64:3.1 \</span><br><span class="line">--config=/etc/kubernetes/kubeletConfig \</span><br><span class="line">--logtostderr=false --log-dir=/var/log/kubernetes --v=2&quot;</span><br></pre></td></tr></table></figure>

<p>可以看到变化比较大，简洁很多。而且KubeletConfiguration还能做成configMap，这样就能重用。</p>
<h1 id="kubelet-认证"><a href="#kubelet-认证" class="headerlink" title="kubelet 认证"></a>kubelet 认证</h1><p>通过上面的方式改变成kubelet参数文件后，有个很大的变化是authentication.anonymous，这个参数默认是true也就是允许匿名。在旧的命令行参数方式没什么问题，但是现在在k8s master上运行kubectl logs或者kubectl exec就会出现报错</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes,subresource=proxy)</span><br></pre></td></tr></table></figure>

<p>明显看出匿名权限问题，网上也有一些解决方法是直接给system:anonymous赋权的，但是毕竟生产环境这样做存在隐患，解决方法是禁用匿名认证，创建一个x509证书认证。<br>怎么创建证书就不说了，配置x509文件的关键是在kubelet配置ca证书，在apiserver配置client证书，然后给证书用户赋权。<a href="https://k8smeetup.github.io/docs/admin/kubelet-authentication-authorization/" target="_blank" rel="noopener">官方说明</a><br>kubelet配置：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">authentication:</span><br><span class="line">  anonymous:</span><br><span class="line">    enabled: false</span><br><span class="line">  x509:</span><br><span class="line">    clientCAFile: /etc/kubernetes/pki/kubelet-ca.pem</span><br></pre></td></tr></table></figure>

<p>apiserver配置：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">--kubelet-client-certificate=/etc/kubernetes/pki/kubelet-client.pem</span><br><span class="line">--kubelet-client-key=/etc/kubernetes/pki/kubelet-client-key.pem</span><br></pre></td></tr></table></figure>

<p>证书用户是kubeletadmin，rbac赋权：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">apiVersion: rbac.authorization.k8s.io/v1</span><br><span class="line">kind: ClusterRole</span><br><span class="line">metadata:</span><br><span class="line">  name: kubelet-admin</span><br><span class="line">rules:</span><br><span class="line">- apiGroups: [&quot;&quot;]</span><br><span class="line">  resources: [&quot;nodes/proxy&quot;,&quot;nodes/stats&quot;,&quot;nodes/log&quot;,&quot;nodes/spec&quot;,&quot;nodes/metrics&quot;]</span><br><span class="line">  verbs: [&quot;*&quot;]</span><br><span class="line">---</span><br><span class="line">apiVersion: rbac.authorization.k8s.io/v1</span><br><span class="line">kind: ClusterRoleBinding</span><br><span class="line">metadata:</span><br><span class="line">  name: kubelet-admin</span><br><span class="line">roleRef:</span><br><span class="line">  apiGroup: rbac.authorization.k8s.io</span><br><span class="line">  kind: ClusterRole</span><br><span class="line">  name: kubelet-admin</span><br><span class="line">subjects:</span><br><span class="line">- apiGroup: rbac.authorization.k8s.io</span><br><span class="line">  kind: User</span><br><span class="line">  name: kubeletadmin</span><br></pre></td></tr></table></figure>

<p>这样就配置好kubelet认证，然后还有授权，默认是AlwaysAllow，这里不改了。</p>
  
	</div>
		<footer class="article-footer clearfix">

  <div class="article-tags">
  
  <span></span> <a href="/tags/kubernetes/">kubernetes</a>
  </div>


<div class="article-categories">
  <span></span>
  <a class="article-category-link" href="/categories/容器架构/">容器架构</a>
</div>



<div class="article-share" id="share">

  <div data-url="http://www.lstop.pub/2020/05/19/kubelet-config和认证/" data-title="kubelet config和认证 | Zong&#39;s blog" data-tsina="" class="share clearfix">
  </div>

</div>
</footer>   	       
	</article>
	
<nav class="article-nav clearfix">
 
 <div class="prev" >
 <a href="/2020/08/28/elasticsearch备份到hdfs/" title="elasticsearch备份到hdfs">
  <strong>PREVIOUS:</strong><br/>
  <span>
  elasticsearch备份到hdfs</span>
</a>
</div>


<div class="next">
<a href="/2020/01/08/ubuntu18下的coredns踩坑/"  title="ubuntu18下的coredns踩坑">
 <strong>NEXT:</strong><br/> 
 <span>ubuntu18下的coredns踩坑
</span>
</a>
</div>

</nav>

	
</div>  
      <div class="openaside"><a class="navbutton" href="#" title="Show Sidebar"></a></div>

  <div id="toc" class="toc-aside">
  <strong class="toc-title">Contents</strong>
  <ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#Kubelet-Configuration"><span class="toc-number">1.</span> <span class="toc-text">Kubelet Configuration</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#kubelet-认证"><span class="toc-number">2.</span> <span class="toc-text">kubelet 认证</span></a></li></ol>
  </div>

<div id="asidepart">
<div class="closeaside"><a class="closebutton" href="#" title="Hide Sidebar"></a></div>
<aside class="clearfix">

  
<div class="tagslist">
	<p class="asidetitle">Tags</p>
		<ul class="clearfix">
		
			<li><a href="/tags/Airtest/" title="Airtest">Airtest<sup>1</sup></a></li>
		
			<li><a href="/tags/DNS/" title="DNS">DNS<sup>1</sup></a></li>
		
			<li><a href="/tags/GitLab/" title="GitLab">GitLab<sup>1</sup></a></li>
		
			<li><a href="/tags/K8s/" title="K8s">K8s<sup>8</sup></a></li>
		
			<li><a href="/tags/Linux/" title="Linux">Linux<sup>1</sup></a></li>
		
			<li><a href="/tags/MongoDB/" title="MongoDB">MongoDB<sup>2</sup></a></li>
		
			<li><a href="/tags/OpenWrt/" title="OpenWrt">OpenWrt<sup>1</sup></a></li>
		
			<li><a href="/tags/Python/" title="Python">Python<sup>2</sup></a></li>
		
			<li><a href="/tags/RabbitMQ/" title="RabbitMQ">RabbitMQ<sup>1</sup></a></li>
		
			<li><a href="/tags/calico/" title="calico">calico<sup>1</sup></a></li>
		
			<li><a href="/tags/cdn/" title="cdn">cdn<sup>1</sup></a></li>
		
			<li><a href="/tags/docker/" title="docker">docker<sup>3</sup></a></li>
		
			<li><a href="/tags/docker-registry/" title="docker registry">docker registry<sup>1</sup></a></li>
		
			<li><a href="/tags/elasticsearch/" title="elasticsearch">elasticsearch<sup>3</sup></a></li>
		
			<li><a href="/tags/elk/" title="elk">elk<sup>3</sup></a></li>
		
			<li><a href="/tags/k8s/" title="k8s">k8s<sup>3</sup></a></li>
		
			<li><a href="/tags/kubernetes/" title="kubernetes">kubernetes<sup>1</sup></a></li>
		
			<li><a href="/tags/nginx/" title="nginx">nginx<sup>1</sup></a></li>
		
			<li><a href="/tags/python/" title="python">python<sup>1</sup></a></li>
		
			<li><a href="/tags/tomcat/" title="tomcat">tomcat<sup>1</sup></a></li>
		
		</ul>
</div>


  <div class="linkslist">
  <p class="asidetitle">Links</p>
    <ul>
      <li><a href="http://www.v2ex.com/?r=zong400" target="_blank" title="V2EX">V2EX</a></li>
      <li><a href="http://hexo.io" target="_blank" title="Hexo">Hexo</a></li>
	  <li><a href="https://promotion.aliyun.com/ntms/yunparter/invite.html?userCode=s0bh6uzq" target="_blank" title="阿里云">阿里云</a></li>
	  <li><a href="https://cloud.tencent.com/redirect.php?redirect=1014&cps_key=5bd9deb84d4d9f34b65fb934e12d03e3&from=console" target="_blank" title="腾讯云">腾讯云</a></li>
    </ul>
</div>


</aside>
</div>
    </div>
    <footer><div id="footer" >
	
	
	<div class="social-font" class="clearfix">
		
		
		
		
	</div>
		<p class="copyright">Hosted by <a href="https://pages.coding.me/" target="_blank" title="Coding Pages">Coding Pages</a></p>
		<p class="copyright">Powered by <a href="http://hexo.io" target="_blank" title="hexo">hexo</a> and Theme by <a href="https://github.com/wizicer/iceman" target="_blank" title="Iceman">Iceman</a> © 2020 
		
		<a href="http://www.lstop.pub" target="_blank" title="Zong">Zong</a>
		
		</p>
</div>
</footer>
    <script src="//cdn.staticfile.org/jquery/2.1.0/jquery.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){ 
  $('.navbar').click(function(){
    $('header nav').toggleClass('shownav');
  });
  var myWidth = 0;
  function getSize(){
    if( typeof( window.innerWidth ) == 'number' ) {
      myWidth = window.innerWidth;
    } else if( document.documentElement && document.documentElement.clientWidth) {
      myWidth = document.documentElement.clientWidth;
    };
  };
  var m = $('#main'),
      a = $('#asidepart'),
      c = $('.closeaside'),
      o = $('.openaside');
  $(window).resize(function(){
    getSize(); 
    if (myWidth >= 1024) {
      $('header nav').removeClass('shownav');
    }else
    {
      m.removeClass('moveMain');
      a.css('display', 'block').removeClass('fadeOut');
      o.css('display', 'none');
      
      $('#toc.toc-aside').css('display', 'none');
        
    }
  });
  c.click(function(){
    a.addClass('fadeOut').css('display', 'none');
    o.css('display', 'block').addClass('fadeIn');
    m.addClass('moveMain');
  });
  o.click(function(){
    o.css('display', 'none').removeClass('beforeFadeIn');
    a.css('display', 'block').removeClass('fadeOut').addClass('fadeIn');      
    m.removeClass('moveMain');
  });
  $(window).scroll(function(){
    o.css("top",Math.max(80,260-$(this).scrollTop()));
  });
});
</script>

<script type="text/javascript">
$(document).ready(function(){ 
  var ai = $('.article-content>iframe'),
      ae = $('.article-content>embed'),
      t  = $('#toc'),
      h  = $('article h2')
      ah = $('article h2'),
      ta = $('#toc.toc-aside'),
      o  = $('.openaside'),
      c  = $('.closeaside');
  if(ai.length>0){
    ai.wrap('<div class="video-container" />');
  };
  if(ae.length>0){
   ae.wrap('<div class="video-container" />');
  };
  if(ah.length==0){
    t.css('display','none');
  }else{
    c.click(function(){
      ta.css('display', 'block').addClass('fadeIn');
    });
    o.click(function(){
      ta.css('display', 'none');
    });
    $(window).scroll(function(){
      ta.css("top",Math.max(140,320-$(this).scrollTop()));
    });
  };
});
</script>


<script type="text/javascript">
$(document).ready(function(){ 
  var $this = $('.share'),
      url = $this.attr('data-url'),
      encodedUrl = encodeURIComponent(url),
      title = $this.attr('data-title'),
      tsina = $this.attr('data-tsina');
  var html = [
  '<a href="#" class="overlay" id="qrcode"></a>',
  '<div class="qrcode clearfix"><span>扫描二维码分享到微信朋友圈</span><a class="qrclose" href="#share"></a><strong>Loading...Please wait</strong><img id="qrcode-pic" data-src="http://s.jiathis.com/qrcode.php?url=' + encodedUrl + '"/></div>',
  '<a href="#textlogo" class="article-back-to-top" title="Top"></a>',
  '<a href="https://www.facebook.com/sharer.php?u=' + encodedUrl + '" class="article-share-facebook" target="_blank" title="Facebook"></a>',
  '<a href="#qrcode" class="article-share-qrcode" title="QRcode"></a>',
  '<a href="https://twitter.com/intent/tweet?url=' + encodedUrl + '" class="article-share-twitter" target="_blank" title="Twitter"></a>',
  '<a href="http://service.weibo.com/share/share.php?title='+title+'&url='+encodedUrl +'&ralateUid='+ tsina +'&searchPic=true&style=number' +'" class="article-share-weibo" target="_blank" title="Weibo"></a>',
  '<span title="Share to"></span>'
  ].join('');
  $this.append(html);
  $('.article-share-qrcode').click(function(){
    var imgSrc = $('#qrcode-pic').attr('data-src');
    $('#qrcode-pic').attr('src', imgSrc);
    $('#qrcode-pic').load(function(){
        $('.qrcode strong').text(' ');
    });
  });
});     
</script>









  </body>
</html>

